Augmenting external control arms using Bayesian borrowing: a case study in first-line non-small cell lung cancer.

Aim: This study aimed to improve comparative effectiveness estimates and discuss challenges encountered through the application of Bayesian borrowing (BB) methods to augment an external control arm (ECA) constructed from real-world data (RWD) using historical clinical trial data in first-line non-small-cell lung cancer (NSCLC). Materials & methods: An ECA for a randomized controlled trial (RCT) in first-line NSCLC was constructed using ConcertAI Patient360™ to assess chemotherapy with or without cetuximab, in the bevacizumab-inappropriate subpopulation. Cardinality matching was used to match patient characteristics between the treatment arm (cetuximab + chemotherapy) and ECA. Overall survival (OS) was assessed as the primary outcome using Cox proportional hazards (PH). BB was conducted using a static power prior under a Weibull PH parameterization with borrowing weights from 0.0 to 1.0 and augmentation of the ECA from a historical control trial. Results: The constructed ECA yielded a higher overall survival (OS) hazard ratio (HR) (HR = 1.53; 95% CI: 1.21-1.93) than observed in the matched population of the RCT (HR = 0.91; 95% CI: 0.73-1.13). The OS HR decreased through the incorporation of BB (HR = 1.30; 95% CI: 1.08-1.54, borrowing weight = 1.0). BB was applied to augment the RCT control arm via a historical control which improved the precision of the observed HR estimate (1.03; 95% CI: 0.86-1.22, borrowing weight = 1.0), in comparison to the matched population of the RCT alone. Conclusion: In this study, the RWD ECA was unable to successfully replicate the OS estimates from the matched population of the selected RCT. The inability to replicate could be due to unmeasured confounding and variations in time-periods, follow-up and subsequent therapy. Despite these findings, we demonstrate how BB can improve precision of comparative effectiveness estimates, potentially aid as a bias assessment tool and mitigate challenges of traditional methods when appropriate external data sources are available.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 15.

In this latest update we discuss real-world evidence (RWE) guidance from the leading oncology professional societies, the American Society of Clinical Oncology and the European Society for Medical Oncology, and the PRINCIPLED practical guide on the design and analysis of causal RWE studies.

Propensity score methods for comparative-effectiveness analysis: A case study of direct oral anticoagulants in the atrial fibrillation population.

OBJECTIVE: To explore methodological challenges when using real-world evidence (RWE) to estimate comparative-effectiveness in the context of Health Technology Assessment of direct oral anticoagulants (DOACs) in Scotland. METHODS: We used linkage data from the Prescribing Information System (PIS), Scottish Morbidity Records (SMR) and mortality records for newly anticoagulated patients to explore methodological challenges in the use of Propensity score (PS) matching, Inverse Probability Weighting (IPW) and covariate adjustment with PS. Model performance was assessed by standardised difference. Clinical outcomes (stroke and major bleeding) and mortality were compared for all DOACs (including apixaban, dabigatran and rivaroxaban) versus warfarin. Patients were followed for 2 years from first oral anticoagulant prescription to first clinical event or death. Censoring was applied for treatment switching or discontinuation. RESULTS: Overall, a good balance of patients' covariates was obtained with every PS model tested. IPW was found to be the best performing method in assessing covariate balance when applied to subgroups with relatively large sample sizes (combined-DOACs versus warfarin). With the IPTW-IPCW approach, the treatment effect tends to be larger, but still in line with the treatment effect estimated using other PS methods. Covariate adjustment with PS in the outcome model performed well when applied to subgroups with smaller sample sizes (dabigatran versus warfarin), as this method does not require further reduction of sample size, and trimming or truncation of extreme weights. CONCLUSION: The choice of adequate PS methods may vary according to the characteristics of the data. If assumptions of unobserved confounding hold, multiple approaches should be identified and tested. PS based methods can be implemented using routinely collected linked data, thus supporting Health Technology decision-making.

Is Real-World Evidence Used in P&T Monographs and Therapeutic Class Reviews?

BACKGROUND: Payers are faced with making coverage and reimbursement decisions based on the best available evidence. Often these decisions apply to patient populations, provider networks, and care settings not typically studied in clinical trials. Treatment effectiveness evidence is increasingly available from electronic health records, registries, and administrative claims. However, little is known about when and what types of real-world evidence (RWE) studies inform pharmacy and therapeutic (P&T) committee decisions. OBJECTIVE: To evaluate evidence sources cited in P&T committee monographs and therapeutic class reviews and assess the design features and quality of cited RWE studies. METHODS: A convenience sample of representatives from pharmacy benefit management, health system, and health plan organizations provided recent P&T monographs and therapeutic class reviews (or references from such documents). Two investigators examined and grouped references into major categories (published studies, unpublished studies, and other/unknown) and multiple subcategories (e.g., product label, clinical trials, RWE, systematic reviews). Cited comparative RWE was reviewed to assess design features (e.g., population, data source, comparators) and quality using the Good ReseArch for Comparative Effectiveness (GRACE) Checklist. RESULTS: Investigators evaluated 565 references cited in 27 monographs/therapeutic class reviews from 6 managed care organizations. Therapeutic class reviews mostly cited published clinical trials (35.3%, 155/439), while single-product monographs relied most on manufacturer-supplied information (42.1%, 53/126). Published RWE comprised 4.8% (21/439) of therapeutic class review references, and none (0/126) of the monograph references. Of the 21 RWE studies, 12 were comparative and assessed patient care settings and outcomes typically not included in clinical trials (community ambulatory settings [10], long-term safety [8]). RWE studies most frequently were based on registry data (6), conducted in the United States (6), and funded by the pharmaceutical industry (5). GRACE Checklist ratings suggested the data and methods of these comparative RWE studies were of high quality. CONCLUSIONS: RWE was infrequently cited in P&T materials, even among therapeutic class reviews where RWE is more readily available. Although few P&T materials cited RWE, the comparative RWE studies were generally high quality. More research is needed to understand when and what types of real-world studies can more routinely inform coverage and reimbursement decisions. DISCLOSURES: This project was funded by the National Pharmaceutical Council. Hurwitz, Brown, Peters, and Malone have nothing to disclose. Graff is employed by the National Pharmaceutical Council Part of this study was presented as a poster presentation at the AMCP Managed Care & Specialty Pharmacy 2016 Annual Meeting; April 19-22, 2016; San Francisco, CA. Study concept and design were primarily contributed by Malone and Graff, along with Hurwitz and Brown. All authors participated in data collection, and data interpretation was performed by Malone, Hurwitz, and Graff, with assistance from Brown and Peters. The manuscript was written primarily by Hurwitz and Malone, along with Graff, Brown, and Peters, and revised by Malone, Brown, Peters, Hurwitz, and Graff.

The use of real-world evidence in ICER's scoping process and clinical evidence assessments.

BACKGROUND: There has been growing interest in using real-world evidence (RWE) for health technology assessment (HTA) in the United States. The Institute for Clinical and Economic Review (ICER) is an independent U.S.-based HTA organization that focuses primarily on pharmaceuticals. RWE is used to inform ICER's scoping and comparative clinical effectiveness (CCE) assessments, but the extent to which it is used has not been quantified. OBJECTIVE: To systematically evaluate use of RWE in the scoping and CCE assessment sections of the ICER HTA reports on pharmaceuticals. METHODS: We reviewed all ICER reports of pharmaceuticals published between January 2014 and June 2019. We examined the average number of instances and the proportion of RWE use in the scoping documents to inform the population, intervention, comparator, outcome, setting, or timing (PICOTS) elements of the appraisal. We also examined the average number of instances and the proportion of RWE use in the CCE assessments to inform effectiveness, safety, or treatment patterns. Finally, we evaluated use of RWE in clinical guidelines that were cited in the CCE assessments. RESULTS: In ICER scoping documents, the mean (SD) number of instances of RWE use was 3.8 (3.7) per document (55% for outcomes, 20% for population, 14% for comparator, 11% for intervention, and 0% for timing and setting). In ICER CCE assessments, the mean (SD) number of instances per assessment was 0.7 (0.5) per drug (53% for effectiveness, 44% for safety, and 3% for treatment patterns). In clinical guidelines used in ICER reports, the mean (SD) number of instances of RWE use was 1.6 (2.3) per drug per guideline (41% for effectiveness, 30% for safety, and 29% for treatment patterns). CONCLUSIONS: RWE was frequently used in the ICER scoping process, particularly to inform selection of outcomes. RWE was used infrequently in ICER CCE assessments, while more often used to inform effectiveness, safety, and treatment patterns in relevant clinical guidelines. There are opportunities to increase the use of RWE in U.S. HTA processes. DISCLOSURES: This study was supported by the Health Tech Fund, University of Washington School of Pharmacy, which was created through unrestricted support from several health care industry companies. Veenstra and Carlson report grant support from the Institute for Clinical and Economic Review outside the submitted work. Carlson reports personal fees from Bayer, Allergan, and Galderma outside the submitted work. Jiao, Lee, and Devine report no support outside the submitted work.

Genome Diagnostics: Novel Strategies for Measuring Value.

Genetic testing technology is rapidly evolving with the growth of personalized medicine. While test evaluation typically relies on laboratory measures of performance, tests can be costly and analytically and ethically complex. A more fulsome consideration of value is warranted to inform adoption and appropriate use. Herein we describe a methodology for developing novel clinician- and patient-reported measures of clinical and personal utility, aiming to capture the informational value of genome diagnostic tests. Adhering to core measurement science principles and standards, our 4-step process includes (1) tool development through scoping reviews and stakeholder interviews and surveys; (2) tool validation through prospective cohort studies to establish construct validity, inter- and intra-rater reliability; (3) tool application using comparative effectiveness assessment to gauge the comparative value of different types of genetic tests; and (4) tool dissemination, leveraging existing partnerships with international stakeholders to spur additional validation studies, comparative effectiveness research, cost-effectiveness analysis, and evidence-informed policy. A scoping review of the clinical utility literature informed the development of a preliminary 25-item index. Qualitative interviews with 35 clinicians further informed the definition of our utility construct, item content, and item importance. Stakeholder surveys with 113 clinicians enabled further feedback on item content, importance, sensibility, response, and scoring options. An 18-item tool, the "Clinician-reported Genetic testing Utility InDEx" (C-GUIDE), is now undergoing validation, while development work on the patient-reported measure of utility is underway. A methodologically innovative approach to the development of stakeholder-informed and clinimetrically sound measures of value for personalized medicine tests will assist technology users and decision makers globally. DISCLOSURES: This work was supported by the Canadian Institutes of Health Research Operating Grant (#PJT-152880) and the PhRMA Foundation Challenge Award. Publication of the study methodology or findings generated therein was not contingent on the sponsor's approval or censorship of the manuscript. The authors have nothing to disclose. Results from this study were presented as a poster at the 40th Annual North American Meeting of the Society for Medical Decision Making; October 14, 2018; Montreal, QC; the Annual Meeting of the American Society of Human Genetics; October 18, 2018; San Diego, CA; and as an oral presentation at the Annual Meeting of the Canadian Association for Health Services and Policy Research; May 31, 2018; Montreal, QC.

Evaluation of bottom-up interventions targeting community-dwelling frail older people in Belgium: methodological challenges and lessons for future comparative effectiveness studies.

BACKGROUND: Optimizing the organization of care for community-dwelling frail older people is an important issue in many Western countries. In Belgium, a series of complex, innovative, bottom-up interventions was recently designed and implemented to help frail older people live at home longer. As the effectiveness of these interventions may vary between different population groups according to their long-term care needs, they must be evaluated by comparison with a control group that has similar needs. METHODS: The goal was to identify target groups for these interventions and to establish control groups with similar needs and to explore, per group, the extent to which the utilization of long-term care is matched to needs. We merged two databases: a clinical prospective database and the routine administrative database for healthcare reimbursements. Through Principal Component Analysis followed by Clustering, the intervention group was first stratified into disability profiles. Per profile, comparable control groups for clinical variables were established, based on propensity scores. Using chi-squared tests and logistic regression analysis, long-term care utilization at baseline was then compared per profile and group studied. RESULTS: Stratification highlighted five disability profiles: people with low-level limitations; people with limitations in instrumental activities of daily life and low-level of cognitive impairment; people with functional limitations; people with functional and cognitive impairments; and people with functional, cognitive, and behavioral problems. These profiles made it possible to identify long-term care needs. For instance, at baseline, those who needed more assistance with hygiene tasks also received more personal nursing care (P < 0.05). However, there were some important discrepancies between the need for long-term care and its utilization: while 21% of patients who were totally dependent for hygiene tasks received no personal nursing care, personal nursing care was received by 33% of patients who could perform hygiene tasks. CONCLUSIONS: The disability profiles provide information on long-term care needs but not on the extent to which those needs are met. To assess the effectiveness of interventions, controls at baseline should have similar disability profiles and comparable long-term care utilization. To allow for large comparative effectiveness studies, these dimensions should ideally be available in routine databases.

Study design choices for evaluating the comparative safety of diabetes medications: An evaluation of pioglitazone use and risk of bladder cancer in older US adults with type-2 diabetes.

AIM: The aim of the study was to empirically demonstrate the effect of varying study designs when evaluating the safety of pioglitazone in treating bladder cancer. METHODS: We identified Medicare beneficiaries above 65 years of age with diabetes between 2008 and 2015 and with classified exposure (at least two claims within 180 days) to glucose-lowering drugs (GLD), pioglitazone or another drug. The effects of varying the following study design parameters on bladder cancer risk were assessed: use of a new vs existing drug, choice of referent (all non-users and users of GLDs, non-insulin GLDs and DPP-4s) and whether or not censoring accounted for treatment change. We used the Cox proportional hazards model to obtain adjusted HRs and 95% CIs. RESULTS: We included 1,510,212 patients classified as pioglitazone users (N = 135,188) or non-users (N = 1,375,024). Users had more diabetic complications than non-users, but fewer than insulin users. The HR ranged from 1.10 (1.01-1.20) to 1.13 (0.99-1.29) when censoring ignored treatment change, suggesting a weak association or none between pioglitazone and bladder cancer, probably under-estimating risk. However, the HR was 1.20 (1.01-1.42) when cohorts were restricted to new users, censored upon treatment change, and when DPP-4 was used as the referent, suggesting an increased risk of bladder cancer associated with pioglitazone. CONCLUSIONS: The continued demand for new GLDs indicates the need for more robust observational methods to improve the value of generating real-world evidence in equipping clinicians to make informed prescribing decisions. Although there is no one-size-fits-all approach, we recommend active comparator new user study designs that compare therapeutically equivalent drugs and account for treatment changes during follow-up to present the least biased comparative safety estimates.

Beyond a Seat at the Table: The Added Value of Family Stakeholders to Improve Care, Research, and Education in Neonatology.

OBJECTIVES: To analyze activities involving veteran resource parents and patients in a family partnership program; their perspectives were also explored. STUDY DESIGN: The multiple roles assumed by family stakeholders in neonatal initiatives were reviewed. Quality control questionnaires were distributed to resource parents and patients and providers who worked with them. Mixed methods were used to analyze results. RESULTS: Thirty resource parents and patients were involved in a total of 653 activities related to clinical care (n = 413), teaching (n = 31), and research (n = 209); 7 initiatives were described to illustrate the positive impact of family stakeholders on clinical care, teaching, and/or research. Resource parents and patients had different degrees and intensity of involvement: all were involved in low-risk initiatives and 9 in more complex activities. In the questionnaire, family stakeholders all described positive impacts associated with their participation and benefits to themselves, such as meaning making. Three resource parents reported traumatic memories that occurred during medical simulations. The majority of providers report that resource parents and patients improved their projects, but some also report this new collaboration is complex. CONCLUSIONS: Although stakeholder participation increasingly is recommended, practical knowledge and the impact of their participation is scarce. Having several resource parents and patients bring their contributions may be more valuable than a few "expert stakeholders." Recruiting and orienting resource parents and patients toward different types of activities should take into account the complexity and risks of the tasks. Family stakeholders are appreciated and have a positive impact on projects in which they are involved.

Challenges and Opportunities for Using Big Health Care Data to Advance Medical Science and Public Health.

Methodological advancements in epidemiology, biostatistics, and data science have strengthened the research world's ability to use data captured from electronic health records (EHRs) to address pressing medical questions, but gaps remain. We describe methods investments that are needed to curate EHR data toward research quality and to integrate complementary data sources when EHR data alone are insufficient for research goals. We highlight new methods and directions for improving the integrity of medical evidence generated from pragmatic trials, observational studies, and predictive modeling. We also discuss needed methods contributions to further ease data sharing across multisite EHR data networks. Throughout, we identify opportunities for training and for bolstering collaboration among subject matter experts, methodologists, practicing clinicians, and health system leaders to help ensure that methods problems are identified and resulting advances are translated into mainstream research practice more quickly.

Systematic mapping of checklists for assessing transferability.

BACKGROUND: Systematic reviews of research evidence have become an expected basis for decisions about practice guidelines and policy decisions in the health and welfare sectors. Review authors define inclusion criteria to help them determine which studies to search for and include in their reviews. However, these studies may still vary in the extent to which they reflect the context of interest in the review question. While most review authors would agree that systematic reviews should be relevant and useful for decision makers, there appears to be few well known, if any, established methods for supporting review authors to assess the transferability of review findings to the context of interest in the review. With this systematic mapping and content analysis, we aim to identify whether there exists checklists to support review authors in considering transferability early in the systematic review process. The secondary aim was to develop a comprehensive list of factors that influence transferability as discussed in existing checklists. METHODS: We conducted a systematic mapping of checklists and performed a content analysis of the checklist criteria included in the identified checklists. In June 2016, we conducted a systematic search of eight databases to identify checklists to assess transferability of findings from primary or secondary research, without limitations related to publication type, status, language, or date. We also conducted a gray literature search and searched the EQUATOR repository of checklists for any relevant document. We used search terms such as modified versions of the terms "transferability," "applicability," "generalizability," etc. and "checklist," "guideline," "tool," "criteria," etc. We did not include papers that discussed transferability at a theoretical level or checklists to assess the transferability of guidelines to local contexts. RESULTS: Our search resulted in 11,752 titles which were screened independently by two review authors. The 101 articles which were considered potentially relevant were subsequently read by two authors, independently in full text and assessed for inclusion. We identified 31 relevant checklists. Six of these examined transferability of economic evaluations, and 25 examined transferability of primary or secondary research findings in health (n = 23) or social welfare (n = 2). The content analysis is based on the 25 health and social welfare checklists. We identified seven themes under which we grouped categories of checklist criteria: population, intervention, implementation context (immediate), comparison intervention, outcomes, environmental context, and researcher conduct. CONCLUSIONS: We identified a variety of checklists intended to support end users (researchers, review authors, practitioners, etc.) to assess transferability or related concepts. While four of these checklists are intended for use in systematic reviews of effectiveness, we found no checklists for qualitative evidence syntheses or for the field of social welfare practice or policy. Furthermore, none of the identified checklists for review authors included guidance to on how to assess transferability, or present assessments in a systematic review. The results of the content analysis can serve as the basis for developing a comprehensive list of factors to be used in an approach to support review authors in systematically and transparently considering transferability from the beginning of the review process.

A framework for monitoring of new drugs in Sweden.

In order to monitor the net public health benefit of new drugs, especially in the light of recent stepwise approval approaches, there is a need to optimize real-time post-marketing evaluation of new drugs using data collected in routine care. Sweden, with its unique possibilities for observational research, can provide these data. We herein propose a framework for continuous monitoring of the effectiveness, safety, and cost-effectiveness of new drugs, using prospectively determined protocols designed in collaboration between all relevant stakeholders. We believe that this framework can be a useful tool for healthcare authorities and reimbursement agencies in the introduction of new drugs.

Impact of a goal-directed fluid therapy on length of hospital stay and costs of hepatobiliarypancreatic surgery: a prospective observational study.

AIM: The effectiveness of goal-directed fluid therapy (GDFT) algorithms in improving postoperative outcomes has extensively been suggested. Nevertheless, there is a lack of strong evidence regarding both the clinical impact and the cost-effectiveness of the GDFT protocols. The aim of this study is to evaluate the costs of patients undergoing hepatobiliopancreatic surgery when a GDFT protocol is applied. Materials & methods: Consecutive ASA I-III patients undergoing hepatobiliopancreatic surgery were included in this prospective observational study. Depending on device availability, patients were handled either by fluid therapy guided by Vigileo monitor-derived hemodynamic variables (Vigileo-GDFT group) or by standard fluid treatment (standard group). Postoperative length of stay and economic costs were analyzed. RESULTS: In total, 147 patients were included (71 in the Vigileo-GDFT group and 76 in the standard group). The total hospital length of stay was 13 (median, 1st-3rd quartile, 9-20) days for the Vigileo-GDFT group and 14 (8-21) days for the standard group (p = 0.58); no statistically significant differences between the two groups emerged regarding costs and postoperative complications. In both groups, complications were the main contributor to total cost sustained. CONCLUSION: The application of a GDFT algorithm did not reduce the total length of hospital stay and the global costs, which were mainly influenced by the number of complications.

Real-world evidence for coverage decisions: opportunities and challenges.

AIM: To explore current uses of real-world evidence (RWE) in the US healthcare system, summarize key concerns and highlight various opportunities that could be realized through best use of RWE. Materials & methods: Information was gathered via a literature review and interviews to generate a background paper for the 2017 Institute for Clinical and Economic Review Policy Summit meeting. RESULTS: RWE is currently being utilized in drug development decisions, regulatory approval decisions, post-approval monitoring, payer coverage decisions (initial decisions and reassessments) and for outcomes-based contracting. Solutions to key challenges and opportunities for future development are presented. CONCLUSION: Exciting opportunities for the use of RWE exist, yet important reservations remain. Solutions are within reach if effective partnerships between stakeholders can be nurtured.

A framework to guide the optimal development and use of real-world evidence for drug coverage and formulary decisions.

AIM: To provide a framework for optimizing the development and use of real-world evidence (RWE) in drug coverage decisions. Materials & methods: The Institute for Clinical and Economic Review convened a Policy Summit with representatives from 23 payer and life science companies that compose the Institute for Clinical and Economic Review membership. RESULTS: Summit participants helped refine a new conceptual framework that emphasizes the central role of contextual considerations and the evidentiary argument that the RWE is intended to support in designing the process for the development and interpretation of RWE. CONCLUSION: This framework may provide a structured way for pharmaceutical manufacturers and payers to develop a shared understanding of the best way to develop RWE that will ultimately be useful in informing coverage and formulary decisions.

Budget impact analysis of ocriplasmin for the treatment of symptomatic vitreomacular adhesion in the USA.

BACKGROUND: Vitreomacular traction (VMT) treatment options include watchful waiting, vitrectomy and intravitreal ocriplasmin injection (Jetrea®). This analysis used results from the recently completed OASIS randomized clinical trial to evaluate the 2-year budget impact of ocriplasmin injection availability for treatment of Stage I or II VMT without epiretinal membrane formation in a modeled US health plan. MATERIALS & METHODS: VMT prevalence, treatment patterns and disease resolution rates were from literature, a US retinal-specialist survey and the OASIS trial. Medicare payment rates were applied and a national scenario analysis was conducted. RESULTS: With ocriplasmin available, vitrectomy use and complications-related costs decreased. Budget impact of ocriplasmin to the health plan was US$143,599 over 2 years or US$0.0060 per-member per-month. CONCLUSION: Ocriplasmin was projected to be minimally cost-additive at US$0.0060 per-member per-month over 2 years.

The joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making: an interview with Marc Berger.

Marc L Berger, MD, is a retired, part-time consultant. He recently became Chair of the Real World Evidence Advisory Board for SHYFT Analytics. Over a 25-year industry career, Marc has held senior-level positions at Pfizer, Inc., OptumInsight, Eli Lilly and Company, and Merck & Co., Inc. His professional activities have included serving on committees for Center for Medicare and Medicaid Services (CMS), Agency for Healthcare Research and Quality (AHRQ), Patient-Centered Outcomes Research Institute (PCORI), the International Society for Pharmacoeconomics and Outcomes Research (ISPOR), Drug Information Association (DIA), and the editorial advisory boards of several journals. Marc has written or co-written more than 100 peer-reviewed articles, book chapters, and other publications on a range of topics including health services research, outcomes research, health economics, health policy, and the analysis of real-world data.